WHAT IS CLAIMED IS: 



1 . A computer-readable medium having computer executable instructions for 
automatically constructing an image mosaic from a set of images of a scene, said 
5 computer executable instructions comprising: 
inputting a set of images of a scene; 
registering the set of images; 

comparing the set of images to identify areas of difference between the 
images for all images, said areas of difference representing regions of potential 
10 occlusion in each image; 

determining, for each image, whether regions of potential occlusion in 
each image represent actual regions of occlusion, or whether the regions of 
potential occlusion in each image represent regions of non-occlusion, by 
determining a level of discontinuity along an exterior border of each region of 
15 potential occlusion in each image; and 

creating a mosaic image by replacing at least one actual region of 
occlusion in one image from the set of images with corresponding regions of non- 
occlusion from at least one other image from the set of images of the scene. 

20 2. The computer-readable medium of claim 1 wherein registering the images 
comprises at least one of translating the images, rotating the images, scaling the 
images and warping the images, for aligning the images. 

3. The computer-readable medium of claim 1 further comprising cropping the 
25 images after registering the set of images so that all images cover approximately 

the same view of the scene. 

4. The computer-readable medium of claim 1 further comprising balancing 
the set of images. 

30 
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5. The computer-readable medium of claim 4 wherein balancing the images 
comprises histogram averaging of the images based on corresponding targets for 
each image. 

5 6. The computer-readable medium of claim 4 wherein balancing the images 
comprises white balancing the images based on corresponding targets for each 
image. 

7. The computer-readable medium of claim 1 wherein comparing the set of 
10 images to identify areas of difference between the images for all images further 

comprises comparing corresponding blocks of image pixels between each image 
to determine whether the corresponding blocks match. 

8. The computer-readable medium of claim 1 wherein comparing the set of 
15 images to identify areas of difference between the images for all images further 

comprises automatically constructing at least one image mask for identifying non- 
occluded regions and regions of potential occlusion in the set of images. 

9. The computer-readable medium of claim 8 determining whether the 
20 corresponding blocks match further comprises determining whether the 

corresponding blocks match within a predetermined matching threshold. 

10. The computer-readable medium of claim 1 wherein replacing at least one 
actual region of occlusion in one image from the set of images with 

25 corresponding regions of non-occlusion further comprises at least one of 

blending, feathering, and pixel averaging, along the edge of the corresponding 
regions of non-occlusion used to replace any actual regions of occlusion. 

1 1 . The computer-readable medium of claim 1 further comprising a user 
30 interface for excluding one or more actual regions of occlusion from being 

replaced with corresponding regions of non-occlusion. 
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12. A system for removing occlusions from a composite image formed from a 
set of images of a scene, comprising: 

acquiring at least two images of a scene from approximately the same 
viewpoint; 

5 aligning each of the images to a base image selected from the set of 

images; 

identifying areas of potential occlusion in each of the aligned images; 
selecting a seed image from the set of images; 

determining whether each area of potential occlusion in the seed image is 
10 an actual area of occlusion by examining each area of potential occlusion in the 
seed image to determine whether a level of discontinuity along an outer edge of 
each area of discontinuity exceeds a predetermined threshold; 

replacing areas of actual occlusion in the seed image with corresponding 
non-occluded areas from one of the other images in the set to form a composite 
15 image from the seed image. 

13. The system of claim 12 wherein the system is embedded in a digital 
camera. 

20 14. The system of claim 1 2 wherein aligning each of the images to the base 
images comprises applying a geometric transform to each image relative to the 
base image for registering each of the images to the base image. 

1 5. The system of claim 14 further comprising individually cropping each 
25 image so that all images cover approximately the same view of the scene. 

16. The system of claim 12 further comprising balancing the images by 
applying any of histogram averaging of the images based on corresponding 
targets for each image, and white balancing of the images based on 

30 corresponding targets for each image. 
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17. The system of claim 12 wherein identifying areas of potential occlusion in 
each of the aligned images further comprises dividing each image in the set of 
images into a number of pixel blocks, and comparing corresponding pixel blocks 
between each image to determine whether the corresponding blocks match 

5 within a predetermined threshold. 

18. The system of claim 12 wherein identifying areas of potential occlusion in 
each of the aligned images further comprises a pixel-by-pixel comparison of 
corresponding pixels between each image to determine whether the 

10 corresponding pixels match within a predetermined threshold. 

19. The system of claim 12 wherein identifying areas of potential occlusion in 
each of the aligned images further comprises automatically constructing an 
image mask for identifying all regions of potential occlusion in the set of images. 

15 

20. The system of claim 12 wherein replacing areas of actual occlusion in the 
seed image with corresponding non-occluded areas from one of the other images 
in the set further comprises at least one of blending, feathering, and pixel 
averaging an outer edge of the corresponding non-occluded areas with respect 

20 to the seed image. 

21 . A computer-implemented process for removing occlusions from a mosaic 
image created from a set of images of a scene, comprising: 

inputting a set of two or more images of a scene; 
25 aligning each of the images to a base image selected from the set of 

images; 

comparing each of the aligned images to identify areas of potential 
occlusion in each of the aligned images; 

determining a level of discontinuity -along an outer edge of each area of 
30 potential occlusion for each image, said level of discontinuity indicating an area 
of actual occlusion where the level of discontinuity exceeds a predetermined 
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discontinuity threshold, and said level of discontinuity indicating an area of non- 
occlusion where the level of discontinuity is less than the predetermined 
discontinuity threshold; 

creating an image mask for each image, said image masks indicating 
5 areas of occlusion and areas of non-occlusion for each image; and 

using the image mask for each image for creating a mosaic image by 
replacing areas of actual occlusion in one of the images with corresponding 
areas of non-occlusion from one of the other images. 

10 22. The computer-implemented process of claim 21 wherein the computer is 
integral to a digital camera, and wherein the composite image is formed 
automatically from images of a scene captured using the digital camera. 

23. The computer-implemented process of claim 21 wherein replacing areas 
15 of actual occlusion in one of the images with corresponding areas of non- 
occlusion from one of the other images further comprises blending pixels along 
an outer edge of the corresponding areas of non-occlusion with pixels 
surrounding the areas of actual occlusion being replaced. 

20 24. The computer-implemented process of claim 21 wherein replacing areas 
of actual occlusion in one of the images with corresponding areas of non- 
occlusion from one of the other images further comprises feathering pixels along 
an outer edge of the corresponding areas of non-occlusion with pixels 
surrounding the areas of actual occlusion being replaced. 

25 

25. The computer-implemented process of claim 21 wherein replacing areas 
of actual occlusion in one of the images with corresponding areas of non- 
occlusion from one of the other images further comprises averaging pixels along 
an outer edge of the corresponding areas of non-occlusion with pixels 
30 surrounding the areas of actual occlusion being replaced. 
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26. The computer-implemented process of claim 21 further comprising 
balancing the set of images after registering the set of images. 



27. The computer-implemented process of claim 26 wherein balancing the 

5 images comprises histogram averaging of the images based on corresponding 
targets for each image. 

28. The computer-implemented process of claim 26 wherein balancing the 
images comprises white balancing the images based on corresponding targets 

10 for each image. 
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